Variational Learning in Graphical Models and Neural Networks
نویسنده
چکیده
Variational methods are becoming increasingly popular for inference and learning in probabilistic models. By providing bounds on quantities of interest, they offer a more controlled approximation framework than techniques such as Laplace’s method, while avoiding the mixing and convergence issues of Markov chain Monte Carlo methods, or the possible computational intractability of exact algorithms. In this paper we review the underlying framework of variational methods and discuss example applications involving sigmoid belief networks, Boltzmann machines and feed-forward neural networks.
منابع مشابه
Composing graphical models with neural networks for structured representations and fast inference
We propose a general modeling and inference framework that combines the complementary strengths of probabilistic graphical models and deep learning methods. Our model family composes latent graphical models with neural network observation likelihoods. For inference, we use recognition networks to produce local evidence potentials, then combine them with the model distribution using efficient me...
متن کاملNeural Variational Inference and Learning in Undirected Graphical Models
Many problems in machine learning are naturally expressed in the language of undirected graphical models. Here, we propose black-box learning and inference algorithms for undirected models that optimize a variational approximation to the log-likelihood of the model. Central to our approach is an upper bound on the logpartition function parametrized by a function q that we express as a flexible ...
متن کاملVBALD - Variational Bayesian Approximation of Log Determinants
Evaluating the log determinant of a positive definite matrix is ubiquitous in machine learning. Applications thereof range from Gaussian processes, minimum-volume ellipsoids, metric learning, kernel learning, Bayesian neural networks, Determinental Point Processes, Markov random fields to partition functions of discrete graphical models. In order to avoid the canonical, yet prohibitive, Cholesk...
متن کاملOn Structured Variational Approximations
The problem of approximating a probability distribution occurs frequently in many areas of applied mathematics including statistics communication theory machine learning and the theoretical analysis of complex systems such as neural networks Saul and Jordan have recently proposed a powerful method for e ciently ap proximating probability distributions known as structured variational approximati...
متن کاملOn the Origin of Deep Learning
This paper is a review of the evolutionary history of deep learning models. It covers from the genesis of neural networks when associationism modeling of the brain is studied, to the models that dominate the last decade of research in deep learning like convolutional neural networks, deep belief networks, and recurrent neural networks, and extends to popular recent models like variational autoe...
متن کامل